Digital imaging for determining mix ratio of a coating

ABSTRACT

A method performed by an electronic device includes storing a correlation between spectral data values and mix ratio values of components of a mixture. Spectral data is acquired from a coating, of the mixture, that is applied to a substrate. A mix ratio, of the components that are in the coating, is determined from the acquired spectral data based on the stored correlation.

CROSS-REFERENCE TO RELATED APPLICATIONS

This is a continuation-in-part of U.S. application Ser. No. 14/645,623,filed Mar. 12, 2015, which claims priority from U.S. ProvisionalApplication No. 61/951,603, filed on Mar. 12, 2014, both of theseapplications being incorporated herein by reference.

TECHNICAL FIELD

This is related to systems and methods for performing digital imageprocessing related to coatings and displaying, and more particularly, tosystems and methods providing real-time enhanced digital imaging for theprediction, application, and inspection of coatings.

BACKGROUND

In some applications, the resultant thickness of a coating (e.g., apaint) that is applied to a substrate (e.g., the surface of a metalsubstrate) by a user may be critical, or at least important, to providedesired performance (e.g., proper protection of the substrate). Forexample, achieving a specified thickness of an applied coating may becritical to preventing corrosion of a metal substrate used in marineapplications. Self-inspecting coatings are used in applications such as,for example, marine applications and oil and gas pipeline applications.A self-inspecting coating often includes a coating (e.g. liquid orpowder) that provides a visual indication (e.g., visible or invisible tonaked eyes) of coating properties (such as thickness). As an example,the visual indication of the coating properties may be provided as thecoating is applied or after the coating is applied. For example, a colorof the coating can change as the applied thickness changes, inaccordance with an embodiment. In this manner, a user is able to performa certain level of self-inspecting as the user applies the coating. Thatis, the user may visually observe the color of the coating as it isapplied to the substrate in an attempt to determine if the thickness iscorrect. However, the ability of a user to discern variations in color(and, therefore, variations in the coating film) by observing thecoating with the naked eye is limited.

Further limitations and disadvantages of conventional, traditional, andproposed approaches will become apparent to one of skill in the art,through comparison of such systems and methods with embodiments of thepresent invention as set forth in the remainder of the presentapplication with reference to the drawings.

SUMMARY

Systems and methods providing digitally enhanced imaging for theprediction, application, and inspection of coatings are disclosed. Whilemany of the embodiments are described as occurring in “real-time,” itshould be understood that the systems and methods described herein canbe used in real-time as well as with a delay in processing or analyzingan image. A real-time digital imaging and processing device providesreal-time image acquisition, processing, and display of acquired digitalimaging data to allow a user to discern variations (e.g., variations inthe thickness of a self-inspecting coating being applied to thesubstrate) beyond that which can be discerned by observing with thenaked eye. The real-time digital imaging and processing device may alsoprovide pre-coating and post-coating inspection capabilities as well ascoating prediction capabilities.

Additionally various embodiments of systems and methods may providereal-time enhanced digital imaging methods including but not limited tothe use of; calibration, optical lenses, controlled light source,stereoscopy, multi-spectral imaging (e.g., both real-time and stillimage coating inspection via multi-spectral analysis may be ofinterest), digital identification (e.g., using a QR code), location andorientation based services, coatings with designed chromism, stationarydevices, portable devices, remote devices, and wearable devices.Functionality may include but is not limited to; recordability,non-recordable, point detection, mix ratio determination, non-contactcolor matching, metamerism prediction, light source calibration,substrate calibration, coating calibration, display calibration,quantification, definable deviation, definable tolerances, visual filmthickness determination, profile recognition/determination, andnon-contact film thickness metering (i.e., quantified film thickness).

An embodiment of the present invention provides a method. The methodincludes acquiring real-time digital imaging data of a coating beingapplied to a substrate; performing real-time digital image processing onthe real-time digital imaging data to generate enhanced real-timedigital imaging data, wherein the enhanced real-time digital imagingdata provides an enhanced differentiation between colors in the digitalimaging data, and wherein each color in the enhanced real-time digitalimaging data correlates to a thickness of the applied coating; anddisplaying a visual representation of the enhanced real-time digitalimaging data.

Another embodiment of the present invention provides a method. Themethod includes digitally imaging a substrate to be coated to acquiredigital imaging data; digitally processing the digital imaging data toenhance the digital imaging data, thereby generating enhanced digitalimaging data; digitally processing the enhanced digital imaging data toquantify a level of surface characteristics, such as contamination onthe substrate or substrate variations; and displaying at least one of avisual representation of the level of surface contamination and a visualrepresentation of the enhanced digital imaging data.

An embodiment of the present invention provides a method. The methodincludes acquiring real-time digital imaging data of a coating that hasbeen applied to a substrate; performing real-time digital imageprocessing on the real-time digital imaging data to generate enhancedreal-time digital imaging data, and display a visual representation ofthe enhanced real-time digital imaging data wherein the enhancedreal-time digital imaging data provides an enhancement in visualappearance.

An embodiment of the present invention provides a method. The methodincludes acquiring real-time digital imaging data of a coating before ithas been applied to a substrate; performing real-time digital imageprocessing on the real-time digital imaging data to generate enhancedreal-time digital imaging data, and display a visual representation ofthe enhanced real-time digital imaging data wherein the enhancedreal-time digital imaging data provides an enhancement in visualappearance (e.g., inspection of wet paint in production or in can).

A further embodiment of the present invention provides a method. Themethod includes selecting at least one color on a digital imaging andprocessing device; digitally imaging an object (e.g., an interior of aroom) to be painted to acquire digital imaging data using the digitalimaging and processing device; digitally processing the digital imagingdata using the digital imaging and processing device to: segment thedifferent surfaces of the object to be painted from each other in thedigital imaging data, and apply the at least one color to one or more ofthe surfaces in the digital imaging data to generate enhanced digitalimaging data; and displaying a visual representation of the enhanceddigital imaging data on a display screen of the digital imaging andprocessing device.

In an embodiment, a processor-implemented system includes one or moreprocessors. The processors are configured to acquire original image datafrom a coating material applied to a substrate surface, enhance aspectral response differentiation in the original image data to generateenhanced image data, acquire spectral response data associated with oneor more light sources based at least in part on the enhanced image data,acquire coating thickness data of the coating material, and determine aninterrelationship between the spectral response data associated with theone or more light sources and the coating thickness data of the coatingmaterial. One or more non-transitory machine-readable storage media arefor storing the original image data, the enhanced image data, thespectral response data, the coating thickness data, and a data structurefor the interrelationship between the spectral response data associatedwith the one or more light sources and the coating thickness data of thecoating material.

In different embodiments, the spectral response differentiation can beenhanced based at least in part on one or more spectral fingerprints ofone or more non-visible components in the coating material. Theprocessors can be further configured to apply one or more imageprocessing filters to generate the enhanced image data. The imageprocessing filters might include one or more infrared bandwidth spectraloptical filters. The image processing filters might include one or moreultra-violet bandwidth spectral optical filters. The spectral responsedata might include infrared bandwidth spectral responses associated withone or more non-visible components in the coating material. The spectralresponse data might include ultra-violet bandwidth spectral responsesassociated with one or more non-visible components in the coatingmaterial. The processors might determine a formula associated with theinterrelationship between the spectral response data associated with theone or more light sources and the coating thickness data of the coatingmaterial, and calculate the data structure for the interrelationshipusing the formula. The processors might determine the formula using alinear regression method based at least in part on the spectral responsedata associated with the one or more light sources and the coatingthickness data of the coating material. The formula might indicate thata coating thickness is a function of a spectral response, given thesubstrate surface and the one or more light sources.

The processors might further perform a spectral response measurement toacquire the spectral response data, perform a thickness measurement toacquire the coating thickness data, and generate the data structure forthe interrelationship between the spectral response data associated withthe one or more light sources and the coating thickness data of thecoating material, wherein the data structure includes one or morespectral response fields for storing the spectral response data and oneor more coating thickness fields for storing the coating thickness data,the spectral response data being mapped to the coating thickness data inthe data structure.

The processors might acquire test image data from the coating materialapplied to a test surface, enhance a spectral response differentiationin the test image data to generate enhanced test image data, determinetest spectral response data based at least in part on the enhanced testimage data, process a database query that operates over the spectralresponse fields and the coating thickness fields based at least in parton the test spectral response data, and output a test thickness of thecoating material according to the database query.

The processors might acquire the original image data in real-time. Theprocessors might perform one or more image processing operations togenerate the enhanced image data in real-time. The enhanced image datamight be used with coatings-related metadata for alerting andnotification operations associated with out-of-tolerance conditions. Theimage processing operations might include one or more of color mapping,contrast manipulation, histogram equalization, brightness control,masking using spatial convolution kernels, filtering, compression,thresholding, convolution, correlation, segmentation, multi-spectralband ratioing, intensity-hue-saturation (IHS) transformation, spatialconvolution filtering, directional filtering, image subtraction, imagemagnification, layering, focusing, de-focusing, mirroring, and spatialalignment. The processors might display the enhanced image data. Theprocessors might be further configured to scan a code to identify thecoating material and select one or more predetermined image processingoperations and one or more predetermined parameters associated with thecoating material for generating the enhanced image data. The one or morepredetermined parameters associated with the coating material mightinclude one or more calibration factors. The processors might perform acalibration process to correlate one or more substrate surfaces, one ormore coating materials, or one or more light sources to a standard. Theprocessors might be further configured to image the substrate surface tobe coated to acquire substrate imaging data, enhance a spectral responsedifferentiation in the substrate imaging data to generate enhancedsubstrate imaging data, and process the enhanced substrate imaging datato quantify a level of contamination on the substrate surface. Theprocessors might display a visual representation of the level ofcontamination. The processors might display a visual representation ofthe enhanced substrate imaging data. The processors might process theenhanced substrate imaging data to identify one or more types ofcontamination on the substrate surface.

The processors might select inspection presets associated with aparticular type of contaminant before the substrate surface is imaged.The processors might be further configured to process the substrateimaging data to calculate an area of the substrate surface to be coated.The processors might be further configured to image an object whichincludes the substrate surface to be coated to acquire substrate imagingdata, segment a plurality of original surfaces of the object from eachother in the substrate imaging data, the original surfaces including thesubstrate surface, apply visually differentiated indications to theoriginal surfaces in the substrate imaging data to generate enhancedsubstrate imaging data, and display a visual representation of theenhanced substrate imaging data.

The processors might adjust one or more of filters, masks, and layersthat get applied to the substrate imaging data to hone in on aparticular visual indication that is acceptable to a user. Theprocessors might perform real-time adjustment of the one or more offilters, masks, and layers based at least in part on the one or morelight sources. The processors might select and apply a gloss type to thesubstrate imaging data. The processors might process the substrateimaging data to calculate an area of the original surfaces of the objectto be painted. The processors might image the substrate surface to becoated to acquire substrate imaging data, enhance a spectral responsedifferentiation in the substrate imaging data to generate enhancedsubstrate imaging data, and process the enhanced substrate imaging datato determine whether one or more surface preparation operations are tobe performed on the substrate surface. The enhanced image data might beused with coatings-related metadata for alerting and notificationoperations associated with out-of-tolerance conditions.

An example processor-implemented system for determining mix ratiosincludes one or more processors that acquire original image data from acoating material applied to a substrate surface. A spectral responsedifferentiation is enhanced in the original image data to generateenhanced image data. Spectral response data associated with one or morelight sources is acquired based at least in part on the enhanced imagedata. Mix ratio data of the coating material is acquired. Aninterrelationship is determined between the spectral response dataassociated with the one or more light sources and the mix ratio data ofthe coating material. One or more non-transitory machine-readablestorage media store the original image data, the enhanced image data,the spectral response data, the mix ratio data, and a data structure forthe interrelationship between the spectral response data associated withthe one or more light sources and the mix ratio data of the coatingmaterial.

The spectral response differentiation may be enhanced based at least inpart on color pigmentation of one or more components in the coatingmaterial. The spectral response differentiation may be enhanced based onmetamers associated with one or more components in the coating material.The spectral response differentiation may be enhanced based at least inpart on one or more spectral fingerprints of one or more non-visiblecomponents in the coating material. The processors may apply one or moreimage processing filters to generate the enhanced image data. The one ormore image processing filters include one or more infrared bandwidthspectral optical filters. The one or more image processing filtersinclude one or more ultra-violet bandwidth spectral optical filters. Thespectral response data may include infrared bandwidth spectral responsesassociated with one or more non-visible components in the coatingmaterial. The spectral response data might include ultra-violetbandwidth spectral responses associated with one or more non-visiblecomponents in the coating material. The one or more processors maydetermine a formula associated with the interrelationship between thespectral response data associated with the one or more light sources andthe mix ratio data of the coating material, and calculate the datastructure for the interrelationship using the formula. The one or moreprocessors may determine the formula using a linear regression methodbased at least in part on the spectral response data associated with theone or more light sources and the mix ratio data of the coatingmaterial. The formula might indicate that a mix ratio is a function of aspectral response, given the substrate surface and the one or more lightsources. The processors might perform a spectral response measurement toacquire the spectral response data, perform a mix-ratio measurement toacquire the mix ratio data, and generate the data structure for theinterrelationship between the spectral response data associated with theone or more light sources and the mix ratio data of the coatingmaterial, wherein the data structure includes one or more spectralresponse fields for storing the spectral response data and one or moremix-ratio fields for storing the mix ratio data, the spectral responsedata being mapped to the mix ratio data in the data structure. Theprocessors might acquire test image data from the coating materialapplied to a test surface, enhance a spectral response differentiationin the test image data to generate enhanced test image data, determinetest spectral response data based at least in part on the enhanced testimage data, process a database query that operates over the spectralresponse fields and the mix-ratio fields based at least in part on thetest spectral response data, and output a test mix ratio of the coatingmaterial according to the database query. The processors might acquirethe original image data in real-time. The processors might perform oneor more image processing operations to generate the enhanced image datain real-time, wherein the enhanced image data is used withcoatings-related metadata for alerting and notification operationsassociated with out-of-tolerance conditions. The one or more imageprocessing operations include one or more of color mapping, contrastmanipulation, histogram equalization, brightness control, masking usingspatial convolution kernels, filtering, compression, thresholding,convolution, correlation, segmentation, multi-spectral band ratioing,intensity-hue-saturation (IHS) transformation, spatial convolutionfiltering, directional filtering, image subtraction, imagemagnification, layering, focusing, de-focusing, mirroring, and spatialalignment. The processors might scan a code to identify the coatingmaterial and select one or more predetermined image processingoperations and one or more predetermined parameters associated with thecoating material for generating the enhanced image data. The one or morepredetermined parameters might be associated with the coating materialinclude one or more calibration factors. The processors might perform acalibration process to correlate one or more substrate surfaces, one ormore coating materials, or one or more light sources to a standard.

An example processor-implemented system includes one or more processorsconfigured to acquire original image data from a self-inspecting coatingmaterial applied to a substrate surface, enhance a spectral responsedifferentiation in the original image data to generate enhanced imagedata, acquire inherent spectral response data (or designed spectralresponse data) associated with one or more light sources based at leastin part on the enhanced image data, quantify an inherent spectralresponse of the coating material, and determine an interrelationshipbetween the inherent spectral response data (or designed spectralresponse data) associated with the one or more light sources and thequantified inherent spectral response of the coating material. One ormore non-transitory machine-readable storage media might store theoriginal image data, the enhanced image data and the inherent spectralresponse data (or designed spectral response data), and a data structurefor the interrelationship between the inherent spectral response dataassociated with the one or more light sources and the quantifiedinherent spectral response of the coating material. The inherentresponse might corresponds to a thickness of the coating material, or toa mix ratio of the coating material, or to chromism associated with thecoating material.

These and other advantages and novel features of the present invention,as well as details of illustrated embodiments thereof, will be morefully understood from the following description and drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example embodiment of a method of monitoring athickness of a coating on a substrate while applying the coating to thesubstrate;

FIG. 2 illustrates a system block diagram of an example embodiment ofthe real-time digital imaging and processing (RTDIP) device;

FIG. 3 is a flowchart of an example embodiment of the method of FIG. 1of monitoring a thickness of a coating while applying the coating to asubstrate using the real-time digital imaging and processing device ofFIG. 2;

FIG. 4 shows an example embodiment of a first image of a coating on asubstrate before image enhancement as well as an example embodiment of asecond image of the coating on the substrate after image enhancement;

FIG. 5 illustrates several example embodiments of real-time digitalimaging and processing (RTDIP) devices that may be used to perform themethod of FIG. 1 and FIG. 3;

FIG. 6 illustrates an example embodiment of an acquired image of acoating on a substrate after image enhancement and quantization ofcoating thickness;

FIG. 7 illustrates an example embodiment of a code on a coatingcontainer that may be scanned and used to select presets of a real-timedigital imaging and processing (RTDIP) device;

FIG. 8 illustrates an example embodiment of how an individual mayremotely monitor the real-time application of a coating to a substrateusing the method of FIG. 1 and FIG. 3;

FIG. 9 illustrates a plurality of example embodiments of enhanced images(generated by an RTDIP device) of contaminated substrates before acoating is applied;

FIG. 10 is a flowchart of an example embodiment of an inspection methodfor quantifying a level of surface contamination on a substrate to becoated;

FIG. 11 illustrates an example embodiment of a real-time digital imagingand processing device being used to inspect a coated surface anddisplaying an enhanced image showing a coating thickness variation at alocalized spot on the coated surface after the coating has been applied;

FIG. 12 illustrates an example embodiment of a real-time digital imagingand processing (RTDIP) device being used to apply and inspect a coatedsurface using multi-spectrum imaging;

FIG. 13 illustrates a system block diagram of an example embodiment ofthe real-time digital imaging and processing (RTDIP) device of FIG. 12;

FIG. 14 illustrates an example embodiment of several digitally processedimages of a room, each image digitally imposing a different color,showing how the room would appear if painted in different colors;

FIG. 15 illustrates an example embodiment of a digitally processed imageof a room, digitally imposing two colors, showing how the room wouldappear if a first portion of the room were to be painted in a firstcolor and a second portion of the room were to be painted in a secondcolor;

FIG. 16 illustrates an example embodiment of a digitally processed imageof a scene of traffic on a highway, hi-lighting automobiles of aparticular color; and

FIG. 17 shows an example embodiment of a first image of a scene of astore before image processing as well as an example embodiment of asecond image of the same scene of the store after image processing tohi-light a change from the normal scene.

FIGS. 18-21 depict data structures of various embodiments involving themapping of spectral responses with coating thicknesses.

DETAILED DESCRIPTION

Certain embodiments of the systems and methods described herein providereal-time enhanced digital imaging for the prediction, application, andinspection of coatings. Other embodiments of the systems and methodsdescribed herein provide real-time enhanced digital imaging forapplications to law enforcement, security, etc. The embodiments of theinvention as described herein can be applied in real-time or saved forlater review and processing.

Various embodiments of real-time digital imaging and processing (RTDIP)devices and methods described herein provide various combinations ofimage processing techniques to accomplish various application functions,inspection functions, prediction functions, and other security and lawenforcement functions described herein. Various types of imageprocessing techniques may include color mapping, contrast manipulation(enhancement, stretching, linear, non-linear), histogram equalization,brightness control, masking using spatial convolution kernels, filtering(spatial, spectral, temporal, edge enhancement, sharpening, smoothing),compression, thresholding, convolution, correlation, segmentation,multi-spectral band ratioing, intensity-hue-saturation (IHS)transformation, spatial convolution filtering (e.g., directionalfiltering), image subtraction, image magnification, layering, focusing,de-focusing, mirroring, and spatial alignment. Other image processingtechniques may be possible as well. Such image processing techniques maybe implemented in software, hardware, or combinations thereof inaccordance with various embodiments, and may be tuned, calibrated, andpreset for particular modes of operation.

Application Mode

FIG. 1 illustrates an example embodiment of a method of monitoring athickness of a coating 100 on a surface of a substrate 110 whileapplying the coating to the surface of the substrate. As shown in FIG.1, a user 120 is using a spray gun 130 to apply the coating 100 to thesurface of the substrate 110. Other methods of applying the coating arepossible in other embodiments (e.g., using a paint brush that is dippedinto a container containing the coating). In some applications, theresultant thickness of the coating that is applied to the substrate maybe critical, or at least important, to provide proper protection.

Referring to FIG. 1, the spray gun 130 is operatively connected to acoating container 140 that contains a self-inspecting coating (SIC). Inaccordance with an embodiment, a self-inspecting coating includes acoating (e.g. a liquid coating) that provides a visual indication (e.g.,visible or invisible to naked eyes) of thickness. As an example, thevisual indication of the coating properties may be provided as thecoating is applied or after the coating is applied. For example, a colorof the coating can change as the applied thickness changes, inaccordance with an embodiment. In this manner, a user is able to performa certain level of self-inspecting as the user applies the coating. Thatis, the user may visually observe the color of the coating as it isapplied to the substrate in an attempt to determine if the thickness iscorrect.

The ability of a human user to observe the color of a coating (orvariations in the color of the coating across a substrate) with thenaked eye is limited by the actual variation in color that occurs as thethickness of the coating changes and by the visual acuity anddiscernment of the user. However, in the embodiment of FIG. 1, the useris wearing a real-time digital imaging and processing (RTDIP) device 150(e.g., based on Google Glass™) to aid in discerning the colors (and,therefore, the thickness) of the applied coating as the coating isapplied by the user in real-time.

FIG. 2 illustrates a system block diagram of an example embodiment ofthe real-time digital imaging and processing (RTDIP) device 150. Inaccordance with an embodiment, the RTDIP device 150 includes a colorvideo camera 151, a display screen (e.g., a heads-up-display (HUD)) 152,a processing component 153, a user interface 154, a wirelesscommunication component 155, computer memory 156, and software encodedinstructions stored in the computer memory 156 and configured to executeon the processing component 153. The software encoded instructions areconfigured (i.e., programmed) to provide the various functionality(e.g., enhanced color discernment and quantification of coatingthickness) described herein when executed on the processing component153 in cooperative operation with the color video camera 151, thedisplay screen 152, the user interface 154, the wireless communicationcomponent 155, and the computer memory 156 (“computer memory” may referto a physical device or other storage mechanisms such as websites orcloud storage). In accordance with an embodiment, the software encodedinstructions may be in the form of at least an operating system 157 anda real-time enhanced digital imaging (REDI) software application 158stored in the computer memory 156.

The functionality provided by the REDI software application 158 can beconfigured to be fairly comprehensive. For example, the REDI softwareapplication 158 can perform various image enhancement operations, suchas brightness and contrast adjustment, display adjustment, colormapping, channel overlay, noise suppression, segmentation, etc. In someembodiments, the REDI software application 158 performs the imageenhancement operations automatically without user input. In certainembodiments, a user interface is provided to receive user inputs forimage enhancement, and the REDI software application 158 performs theimage enhancement operations in response to the user inputs.

In accordance with alternative embodiments, the processing component 153may be a digital signal processor (DSP) or some other hardware-orientedlogic circuitry. In accordance an embodiment, the RTDIP device is ableto record digital imaging data (video or still images) for subsequentplayback (e.g., recording of acquired imaging data and enhanced imagingdata for comparison). The user interface 154 may include, for example, atouch-screen display, a keyboard, a mouse, voice-activated capability,or some other technology.

In accordance with certain embodiments, the software encodedinstructions are configured (i.e., programmed) to determine aninterrelationship between spectral responses and coating thicknesses.For example, an original image is taken from a self-inspecting materialapplied to a substrate surface. The REDI software application 158performs one or more image enhancement operations to generate anenhanced image. A spectral response with respect to a light source isdetermined from the enhanced image. A measurement of the coatingthickness is performed, and the measured coating thickness is storedtogether with the spectral response in a data structure in the computermemory 156. The above-noted process continues to collect a number ofdata points of spectral responses and coating thicknesses. The softwareencoded instructions are configured (i.e., programmed) to determine aformula indicating the interrelationship between spectral responses andcoating thicknesses based on the collected data points, e.g., using alinear regression method. For example, the formula indicates that acoating thickness is a function of a spectral response, given aparticular substrate surface and a particular light source. Inaccordance with an embodiment, the REDI software application 158 maythen be configured to calculate a coating thickness using the formulabased on a spectral response obtained from an enhanced image.

In accordance with some embodiments, the collected data points ofspectral responses and coating thicknesses may be stored into a databasein the computer memory 156 which maps spectral responses to coatingthicknesses. The REDI software application 158 then performs a databaseinquiry to read out a coating thickness from the database correspondingto a spectral response of the self-inspecting coating material obtainedfrom the enhanced image.

In accordance with some embodiments, a spectral response obtained froman enhanced image may include a recognizable spectral fingerprint of anon-visible component. When the non-visible component is mixed into acoating material, the coating material retains a same visible color, andthe invisible spectral response associated with the spectral fingerprintof the non-visible component may be enhanced and quantified using theREDI technology and correlated to the film thickness.

FIG. 3 is a flowchart of an example embodiment of the method 300 of FIG.1 of monitoring a thickness of a coating while applying the coating to asurface or substrate using the real-time digital imaging and processingdevice 150 of FIG. 2. In step 310 of the method, apply a self-inspectingcoating (SIC) to a surface. As an example of an SIC, a Fast Clad epoxyprimer available from Sherwin Williams can be used as follows. A FastClad epoxy primer is a high solids epoxy amine coating. The pigments areremoved from the primer, and a yellow pigment with low opacityproperties is added to the primer. The primer with the low-opacityyellow pigment is relatively transparent when initially applied to asubstrate, but as the coating becomes thicker, the primer with thepigment becomes more opaque.

It should be understood that different colored pigments can be useddepending upon the application at hand. For example, a yellow pigmentcan be used in situations where the substrate is black. This provides agood color contrast, whereas a black pigment in the coating is noteffective if the underlying substrate is black. Additionally, thecoating may be applied to the surface in any of a number of differentways including using a spray gun or a paint brush.

In step 320, one or more digital images of the SIC being applied to thesurface in real-time are generated to acquire digital imaging data. Forexample, a user wearing the RTDIP device 150 can acquire real-time colordigital video imaging data with the color video camera 151. Inaccordance with an embodiment, the acquired digital image datacorresponds to one or more real-time digital images for the SIC which isapplied to the surface. A digital image may be two or three dimensionaland include one or more color channels. For example, a digital imageincludes a two dimensional grid of pixels, where each pixel isassociated with a set of coordinates and an intensity value (e.g., aninteger between 0 and a maximum intensity value). Higher intensityvalues indicate lighter pixels, and lower intensity values indicatedarker pixels.

In step 330, the digital imaging data is digitally processed inreal-time to enhance a differentiation between colors in the digitalimaging data. For example, the REDI software application 158 running onthe processing component 153 can digitally process the real-time colordigital video imaging data by performing various types of imageprocessing and filtering techniques to generate processed real-timecolor digital video imaging data. The color differences can be made tobe stark to a human user. For example, the color difference can beexplained in different ways, such as through a color space where hue isexpressed as an angle within a cylindrical color space. In such a colorspace, a color difference that would be stark to a human user could be a45 degree difference within the cylindrical coordinate system. In otherwords, colors that are separated by a certain amount of degrees (e.g.,45 degrees or more in certain embodiment) in the cylindrical colorcoordinate color space can provide a satisfactory color difference.However, it should be understood that a stark color contrast may also beachieved with less than 45 degrees.

In accordance with some embodiments, the REDI software application 158performs brightness and contrast adjustment of one or more digitalimages associated with the acquired digital image data. Specifically,the REDI software application 158 selects a number of pixels in adigital image and changes the intensity values of the selected pixelsaccording to a predetermined algorithm. As an example, the REDI softwareapplication 158 maps intensity values of a number of pixels of a digitalimage to display values (e.g., by a linear function). In accordance withcertain embodiments, the REDI software application 158 performs displayadjustments of the one or more digital images associated with theacquired digital image data. For example, pixels with very highintensities and/or very low intensities of a digital image are madevisible, the REDI software application 158 performs non-linear displayadjustments (e.g., Gamma correction, normalization, contrast stretching,histogram equalization, etc.) so that low intensity values become biggerwithout saturating high intensity values.

In accordance with an embodiment, the REDI software application 158performs color mapping of one or more digital images associated with theacquired digital image data. Specifically, the REDI software application158 maps the intensity values of a number of pixels of a digital imageto colors (e.g., using one or more lookup tables).

For example, an intensity value may be mapped to a color which includesthree components corresponding to basic colors, red, green and blue.Different component values of the color indicate different basic colorshades. To enhance color differentiation, the REDI software application158 may select a number of pixels in the digital image and change thecolor values mapped to the selected pixels, so that the basic colorshades of the selected pixels are adjusted. In some embodiments, theREDI software application 158 performs the color enhancement operationsautomatically without user input. In certain embodiments, a userinterface is provided to receive user inputs for color enhancement, andthe REDI software application 158 performs the color enhancementoperations in response to the user inputs. In accordance with anembodiment, the REDI software application 158 performs channel overlayof one or more digital images associated with the acquired digital imagedata. Specifically, the REDI software application 158 creates an overlayof different color channels of a digital image, adjusts the display ofeach channel and transfers the settings from one overlay to another toallow a visual comparison.

In accordance with an embodiment, the REDI software application 158performs noise suppression of one or more digital images associated withthe acquired digital image data. For example, the REDI softwareapplication 158 may apply one or more convolution filters (e.g., a meanfilter, a Gaussian blur filter, an edge enhancing filter, etc.) toreduce the noises in a digital image. In another example, the REDIsoftware application 158 may apply one or more rank filters for noisesuppression, e.g., replacing the intensity values of a number of pixelswith an intensity value of a specifically selected pixel. In accordancewith an embodiment, the REDI software application 158 performssegmentation of one or more digital images associated with the acquireddigital image data to separate one or more objects from the backgroundand separate the objects from each other. For example, a threshold rangeis selected, and all pixels of an object have intensity values withinthe threshold range.

In step 340, the processed digital imaging data is displayed to visuallyshow the enhanced differentiation between the colors in real-time. Forexample, the processing component 153 can format the processed real-timecolor digital video imaging data and send it to the display screen 152for display. In accordance with an embodiment, the display screen 152 isa HUD positioned in front of an eye of the user.

FIG. 4 shows an example embodiment of a first image 410 of a coating ona substrate before image enhancement as well as an example embodiment ofa second image 420 of the coating on the substrate after imageenhancement. The first image 410 is representative of a single image ofthe real-time color digital video imaging data acquired by the colorvideo camera 151 of the RTDIP device 150. The colors in the image 410appear relatively uniform but slightly darker near the middle portion ofthe image 410. When viewing such an unprocessed image, it would bedifficult (if not impossible) for the user to discern any significantcoating thickness variation across the substrate.

The second image 420 is representative of the single image of thereal-time color digital video imaging data acquired by the color videocamera 151 after the image 410 has been processed by the REDI softwareapplication 158 running on the processing component 153. As can be seenin the second image 420, a larger variation in colors appears in thesecond image, providing a much better indication to the user of how thethickness of the coating on the substrate varies. The user can view thisprocessed information on the display screen 152 and use this informationto attempt to smooth out or apply a more uniform coating to thesubstrate. It should be noted at this point that, even though the imagesshown in the figures herein are represented in gray-scale colors,real-world applications can make use of the full spectrum of visiblecolors as digitally represented, for example, by combinations of red(R), green (G), and blue (B) pixels.

FIG. 5 illustrates several example embodiments of real-time digitalimaging and processing (RTDIP) devices that may be used to perform themethod of FIG. 1 and FIG. 3. One embodiment is the wearable RTDIP device150 already discussed herein. Another embodiment is an RTDIP device 510in the form of a laptop computer. A further embodiment is an RTDIPdevice 520 in the form of a mobile telephone (e.g., a “smart phone”).Still another embodiment is an RTDIP device 530 in the form of a tabletcomputer. Yet another embodiment is an RTDIP device 540 having handles541, a light source 542, and a polarized camera lens 543. The lightsource 542 may provide illumination that results in the acquisition ofmore consistent imagery. Furthermore, the polarized lens 543 may serveto reduce or eliminate unwanted reflections or glare in the acquiredimagery. Other devices, other than a polarized lens, may be used toreduce or eliminate unwanted reflections or glare in the acquiredimagery, in accordance with various other embodiments. Each of thesevarious devices may have the components illustrated in FIG. 2, but areeach provided in a different form factor and configuration. Certain formfactors and configurations may be more appropriate for certainapplications. Other form factors and configurations are possible aswell, in accordance with other embodiments.

FIG. 6 illustrates an example embodiment of an acquired image 610 of acoating on a substrate after image enhancement and quantization ofcoating thickness. The image was acquired and displayed using the RTDIPdevice 530 in the form of a tablet computer. The thickness of thecoating varies from left to right (from thinner to thicker) as indicatedby the different colors and by the numeric values (e.g., 5, 10, 15, 20,25) displayed at the bottom of the displayed image 610. Again, it isnoted that, even though the image shown in FIG. 6 is represented ingray-scale colors, real-world applications can make use of the fullspectrum of visible colors as digitally represented, for example, bycombinations of red (R), green (G), and blue (B) pixels.

In accordance with an embodiment, the REDI software application 158 iscalibrated such that the resulting colors may be converted to numericvalues (e.g., 5, 10, 15, 20, 25) being representative of the estimatedthickness (e.g., in millimeters) of the applied coating (quantitativemetering). Each different type of self-inspecting coating (SIC) may haveits own calibration settings to correctly convert the colors of enhancedimage data to numeric thickness values.

FIG. 7 illustrates an example embodiment of a machine-readable code 710on a coating container 140 that may be scanned and used to selectpresets of a real-time digital imaging and processing (RTDIP) device. Inaccordance with an embodiment, the code 710 may be a Quick Response (QR)code (or some other type of bar code) and the RTDIP device 150 may beconfigured to acquire an image of the code 710 using the video camera151, and decode the image of the code 710 using the REDI softwareapplication 158 running on the processing component 153. Alternatively,the RTDIP device 150 may include a separate optical scanner (e.g., alaser scanner) to read the code.

The code 710 identifies the type of SIC in the container 140. Once thecode 710 has been de-coded by the RTDIP device 150 to identify thecoating, the RTDIP device 150 can select the image processingoperations, parameters, and calibration factors that are associated withthe identified coating (i.e., select coating presets). In accordancewith an embodiment, the coating presets associated with the identifiedcoating have been optimized such that processing of acquired real-timecolor digital video imaging data using the coating presets provides goodcolor discernment and/or quantization of coating thickness to the userwhen the enhanced image data is displayed. Optimization or calibrationof the coating presets may take into account the substrate type, thecoating type, the lighting conditions, and additional variables (e.g.,lenses). Calibration is discussed later herein in detail.

As an example, referring to FIG. 1 and FIG. 2, the REDI softwareapplication 158 of the RTDIP device 150 may employ a combination ofspectral filtering techniques, contrast enhancement techniques,histogram equalization techniques, and color mapping techniques in acoating application mode. Such a combination allows the user to morereadily and easily differentiate between the various colors (i.e.,thicknesses) of the self-inspecting coating 100 being applied to thesurface of the substrate 110, in accordance with an embodiment, andprovides to the user a quantitative view of at least the minimum appliedthickness and the maximum applied thickness.

FIG. 8 illustrates an example embodiment of how an individual mayremotely monitor the real-time application of a coating to a substrateusing the method of FIG. 1 and FIG. 3. As described previously herein,the RTDIP device may include a wireless communication component 155. Thewireless communication component 155 may provide WiFi communicationcapability, 3G or LTE communication capability, or some other type ofwireless communication capability through, for example, a communicationnetwork 810. The communication network 810 may be the internet, acellular telephone network, a satellite communication network, somecombination thereof, or some other type of communication network that iscompatible with the wireless communication component 155.

Referring to FIG. 8, a supervisor 820 may be sitting at a computer 830located remotely from where the user 120, who is applying a coating to asubstrate, is located. The enhanced real-time color digital videoimaging data generated by the RTDIP device 150 may be wirelesslytransmitted from the RTDIP device 150, using the wireless communicationcomponent 155, to the remote computer 830 via the communication network810. As a result, the supervisor 820 can monitor the performance of theuser in real-time. If the user seems to be having trouble properlyapplying the coating (e.g., establishing a uniform coating at thespecified thickness), the supervisor may take action to, for example,replace the user with a more qualified person. Other features that maybe provided by an RTDIP device during an application process mayinclude, for example, quality assurance functionality, volumetricquantification of the applied material (e.g., quantified thicknessmultiplied by the calculated dimensions and converted to gallons or someother unit of measure), and hole detection.

Inspection Mode

A substrate to be coated (e.g., a metal substrate) may have rust, salt,dirt or some other contaminating substance on the surface that needs tobe cleaned off before applying a coating material. Even though asubstrate surface may have been “cleaned” and appears to be clean to thenaked eye, an unacceptable level of contamination may still exist on thesubstrate. Such an unacceptable level of contamination may cause asubsequently applied coating to improperly adhere to the surface, thusnot properly protecting the surface. In general, an embodiment of anRTDIP device may be used to detect, identify, quantify, and record thestate of a substrate surface before coating. The inspection mode mayalso be useful for analyzing variations in substrates, for example inporous substrates, or to analyze a pre-treatment that has been appliedto a surface. Analysis of substrate variation or pre-treatment may ormay not use color differences to show variations, but IR light may beused. In accordance with an embodiment, an RTDIP device may be used toimage a surface of a substrate, enhance the image to more clearlydiscern any contaminating substances, and display the enhanced image tothe user.

FIG. 9 illustrates at 902 a plurality of example embodiments of enhancedimages (generated by an RTDIP device) of contaminated substrate surfacesbefore a coating is applied. Each enhanced image of FIG. 9 correspondsto a substrate surface having a different type and amount ofcontaminating substance (e.g., rust, salt, dirt). In some situations,salt may not be visible (in the visible light spectrum) and may requirethe application of an indicator to make the salt visible. However,multi-spectral techniques may be used to detect and visualize salts,that are otherwise not visible in the visible light spectrum, withoutthe use of an applied indicator.

By using a properly configured RTDIP device to provide an enhanced imageof a surface of a substrate before coating, a user may be able toclearly determine if the surface is clean enough to apply a coating. Inaccordance with an embodiment, image processing operations, parameters,and calibration factors (inspection presets) that are associated with acertain type of contamination (e.g., rust, salt, or dirt) may beselected via the user interface 154 of the RTDIP device. In accordancewith an embodiment, the inspection presets associated with a particulartype of contaminant are optimized such that processing of acquiredreal-time color digital video imaging data using the inspection presetsprovides good visual discernment between contaminated andun-contaminated portions of the substrate surface to the user when theenhanced image data is displayed.

In accordance with an embodiment, a type of contaminating substance maybe identified by a user based on a displayed color of the contaminatingsubstance in the enhanced image. For example, rust may be displayed asshades of orange and red. Salt may be displayed as shades of gray. Dirtmay be displayed as shades of brown. A clean, un-contaminated surfacemay appear as white, for example. Furthermore, a level or grade ofsurface preparation may be quantifiable by comparing acquired digitalimaging data to loaded comparative surface preparation standards.

Furthermore, a user may be able to discern not only the presence of aparticular type of contaminating substance but also, at leastqualitatively, an amount of the contaminating substance on any imagedportion of the surface based on color. Also, in accordance with anembodiment, quantitative amounts of a contaminating substance may bedetermined and numerically displayed to the user. For example, apercentage of the surface that is contaminated may be displayed to theuser. This can be accomplished, at least in part, by dividing the numberof pixels in an image showing a contaminating substance (e.g., thenumber of red and orange pixels indicating rust) by the total number ofpixels in the image.

FIG. 10 is a flowchart of an example embodiment of an inspection method1000 for identifying and/or quantifying characteristics of a substrate.Surface characteristics may include, but are not limited to levels ofsurface contamination on a substrate to be coated and surfacevariations. Again, the surface of the substrate may or may not becontaminated with, for example, rust, salt, or dirt. In step 1010 of themethod, a surface of a substrate to be coated is digitally imaged toacquire digital imaging data. For example, a user may use a RTDIP device530 in the form of a tablet computer to image the surface of thesubstrate. In accordance with an embodiment, capturing a single imagemay be sufficient. In step 1020, the digital imaging data is digitallyprocessed to enhance a differentiation between colors in the digitalimaging data, thereby generating enhanced digital imaging data (e.g.,color differentiation could be 30 degrees or more). The differentiationin colors may help discern between contaminated and uncontaminatedpixels in the enhanced digital imaging data, and help discern betweenthe different types of contamination in the enhanced digital imagingdata. Such discernments may not be readily apparent to a user whendirectly viewing the surface of the substrate with the naked eye.

In step 1030, the enhanced digital imaging data is digitally processedto quantify a level of surface contamination on the substrate. Forexample, a numeric value representing a percentage of the imaged surfacethat is contaminated may be generated. As another example, a standarddeviation in pixel color across the enhanced digital imaging data may becomputed and correlated to an amount of contamination on the imagedsurface. In step 1040, a visual representation of the level of surfacecontamination is displayed and, optionally, a visual representation isdisplayed of the enhanced digital imaging data. For example, the levelof surface contamination may be displayed to the user as a numericvalue, and the visual representation of the enhanced digital imagingdata may indicate to the user where on the surface of the substrate mostof the contamination exists. Once the surface of the substrate to becoated is cleaned, the user may perform the inspection method 1000 againto verify that the level of contamination is within acceptable limits.Similar steps could be used to identify and quantify surface variations.

As an example, referring to FIG. 9 and FIG. 10, the REDI softwareapplication 158 of the RTDIP device 530 may employ a combination of edgeenhancement techniques, compression techniques, and thresholdingtechniques in a pre-application inspection mode to allow the user tomore readily and easily determine the presence and qualitative amount ofcontamination on the surface of a substrate to be coated, in accordancewith an embodiment. Furthermore, the REDI software application 158 ofthe RTDIP device 530 may employ a combination of compression, masking,and correlation techniques in the pre-application inspection mode toallow the user to more accurately determine the type of contamination(e.g., rust, salt, dirt), in accordance with an embodiment.

An embodiment of an RTDIP device may be used for inspection aftercoating to enhance problem areas, such as point defects or micro-cracks,where the thickness of the coating is not correct or where the coatingapplied may have had an incorrect mix ratio.

FIG. 11 illustrates an example embodiment of a real-time digital imagingand processing device 530 being used to inspect a coated surface anddisplaying an enhanced image 1110 showing a coating thickness variationat a localized spot on the coated surface after the coating has beenapplied (the applied coating may or may not be dried or cured at thispoint). As can be seen in the central portion of the image 1110, anapparently significant amount of variation in the coating thicknessexists. An inspector can use the enhanced image as proof that thecoating in the localized spot should be corrected (e.g., byre-application of the coating). In other embodiments, the RTDIP devicecould be used to inspect large areas to visualize areas having differentthickness or to visualize areas of the coating that may have otherproblems, such as wrong component mix ratios.

As an example, referring to FIG. 11, the REDI software application 158of the RTDIP device 530 may employ a combination of contrast enhancementtechniques, histogram equalization techniques, color mapping techniques,and magnification techniques in a post-application inspection mode.

In general, an inspector can use the RTDIP device in an inspection modefor quality assurance purposes to detect, identify, quantify (metering),and record a resultant state of a coating after the coating is appliedto a substrate. For example, a standard deviation in pixel color acrossthe enhanced digital imaging data of the coating may be computed andcorrelated to an amount of deviation in coating thickness (or an amountof uniformity of coating thickness) over the substrate.

FIG. 12 illustrates an example embodiment of a real-time digital imagingand processing (RTDIP) device 1200 being used to apply and inspect acoated surface using dual spectrum imaging. FIG. 13 illustrates a systemblock diagram of an example embodiment of the real-time digital imagingand processing (RTDIP) device 1200. Instead of having a single camera151 corresponding to a single electromagnetic frequency spectrum (e.g.,visible light), the RTDIP device 1200 includes a first sensor 1310corresponding to a first electromagnetic frequency spectrum, and asecond sensor 1320 corresponding to a second electromagnetic frequencyspectrum (see FIG. 13). For example, in accordance with an embodiment,the first sensor 1310 is a visible spectrum color video camera and thesecond sensor 1320 is a near infrared (NIR) video sensor.

A user may use the RTDIP device 1200 for coating applications or coatinginspection as previously described herein. However, in the embodiment ofFIG. 12, the RTDIP device 1200 simultaneously captures digital imagingdata in both the first and the second frequency spectrums. FIG. 12 showsa displayed representation of the visible spectrum digital imaging data1210 and a displayed representation of the NIR spectrum digital imagingdata 1220. The RTDIP device 1200 then processes and combines themultiple sources of digital imaging data 1210 and 1220 to form digitalcomposite imaging data 1230. Any of various combinations of the imageprocessing techniques described herein may be used to generate thedigital composite imaging data from the dual sources of digital imagingdata 1210 and 1220.

As an example, referring to FIG. 12 and FIG. 13, the REDI softwareapplication 158 of the RTDIP device may employ a combination of spatialimage alignment techniques, multi-spectral band ratioing techniques,thresholding techniques, and color mapping techniques in apost-application inspection mode.

In an embodiment, the two sensors 1310 and 1320 may be spatially alignedwith each other in the device 1200 such that no processing has to beperformed to align the image data from the two sensors. For example,lenses of the sensors may be positioned and calibrated to make sure thatframes of visible spectrum data are spatially aligned with frames of NIRspectrum data. In accordance with another embodiment, alignmentprocessing is performed to align the raw image data from the two sensorsbefore processing to generate the digital composite imaging data 1230 isperformed. For example, a spatial aligning algorithm may be employed tospatially align or match up pixels of visible spectrum data with pixelsof NIR spectrum data. Such a spatial aligning algorithm may be anythingfrom a sophisticated algorithm that implements state-of-the art aligningtechniques to a simple offset routine that simply applies a known,calibrated offset to the image data in one or more spatial directions.

In accordance with an alternative embodiment, the RTDIP device mayinclude a single multi-spectrum digital sensor, where the single sensoris able to sense both visible-spectrum and non-visible (e.g.,infrared-spectrum) radiation. For example, the single sensor may includea visible-spectrum sensor array interleaved with an infrared-spectrumsensor array, allowing simultaneous capture and formation of bothvisible spectrum and NIR spectrum image data. Alternately, the singlesensor may alternate between capturing visible-spectrum image data andNIR spectrum image data in a time-shared manner on, for example, aframe-to-frame basis. In both cases, a separate set of visible spectrumimage data and NIR spectrum image data are formed and provided to theprocessing component 153. In such a single sensor embodiment, spatialalignment of visible spectrum image data and NIR spectrum image data isinherently achieved.

In accordance with an embodiment, the digital composite imaging data1230 provides better discernment of applied coating thickness thaneither the visible spectrum digital imaging data 1210 alone or thenon-visible (e.g., NIR) spectrum digital imaging data 1220 alone. Thisis because the visible spectrum digital imaging data 1210 providesinformation that the non-visible spectrum digital imaging data 1220 doesnot provide, and vice versa. Therefore, in accordance with anembodiment, it is the digital composite imaging data 1230 that isdisplayed to the user on the display screen 152, instead of the visiblespectrum digital imaging data 1210 or the non-visible spectrum imagingdata 1220. However, as an option, a user may be able to select, via theuser interface 154, which spectral image to display (composite, visible,non-visible). Other non-visible types of electromagnetic frequencyspectrums may be possible to use as well such as, for example, x-ray,ultraviolet, and microwave.

Prediction Mode

In accordance with an embodiment, an RTDIP device may be used to imagean object (e.g., a room) to be painted in real-time (e.g., real-timepanoramic) and process the acquired image data to apply one or morecolors to surface (e.g., the walls, ceiling, or floor) of the object inthe image data.

FIG. 14 illustrates an example embodiment of several digitally processedimages of a room, each image digitally imposing a different color,showing how the room would appear if painted in different colors (e.g.,shown in a split-frame mode). FIG. 14, as shown herein, is limited togray-scale colors. However, in accordance with an embodiment, a fullspectrum of visible light colors may be applied.

As an example, using a RTDIP device (e.g., in the form of a smartphone), a user may select a color from a digital color pallet or fandeck stored in the RTDIP device. The user may then image a room inreal-time (or, optionally, just acquire a single image of the room). Asthe RTDIP device images the room, the RTDIP device processes the imagedata to find boundaries within the image data that define walls, floors,ceilings, and objects within the room. The RTDIP device furtherprocesses the image data to identify pixels associated with the separatewalls, floors, ceilings, and objects within the room. Finally, the RTDIPdevice may apply the selected color to the pixels associated with, forexample, the walls. In accordance with an embodiment, a user may view animage of the room on the RTDIP device and select which surfaces (walls,ceiling, floor) for which to apply the selected color(s).

In this manner, a user may view on the display of the RTDIP device howthe room would look with the walls painted in the selected color. If theuser does not like how the simulated painted walls look, then the usercan select a different color from the digital color pallet or fan deckuntil the user finds an acceptable color. Once the user settles on acolor, the user may order or purchase paint corresponding to that colorand paint the walls accordingly.

Alternatively, instead of selecting a color directly from a digitalcolor pallet or fan deck, the user may adjust various filters, masks,and layers that get applied to the image data to hone in on a color thatis acceptable to the user. Once the user has honed in on an acceptablecolor, a color identifier or code may be generated by the RTDIP devicethat can be used to order paint corresponding to that color.

Also, in accordance with an embodiment, a user may also select a glosstype (e.g., flat, low-sheen, semi-gloss, gloss, full-gloss) in additionto a color. A combination of spectral filtering and IHS transformationmay be used to establish a particular gloss type, in accordance with anembodiment. The embodiment may also display to the user an error ofpredictability (i.e., a range of what the selected color/gloss mightlook like in a room, depending on lighting conditions and otherfactors).

Furthermore, in accordance with an embodiment, the RTDIP device maycalculate the area (e.g., square footage) of the various walls,ceilings, and floors that are identified within the image data andprovide the calculated area information to the user. In this manner, theuser can determine how much paint of a particular color to order. Inaccordance with an embodiment, the RTDIP device uses 3D-sensing andmapping technology such as, for example, technology similar toMicrosoft's KinectFusion™ to map the room in three-dimensions anddetermine the dimensions of individual walls, ceilings, and floors. Fromthese dimensions, the RTDIP device can calculate the areas (e.g., squarefootage). Other technologies for determining the dimensions of a roomare possible as well, in accordance with other embodiments (e.g., lasertechnology, sonar technology). Such dimension-determining techniques mayalso be applied for inspection and application scenarios as well, inaccordance with various embodiments.

FIG. 15 illustrates an example embodiment of a digitally processed imageof a room, digitally imposing two colors, showing how the room wouldappear if a first portion of the room were to be painted in a firstcolor 1510 and a second portion of the room were to be painted in asecond color 1520. Again, a user can select or hone in on two colors anddirect the RTDIP device to apply the two colors to different walls,ceilings, or floors in the image data using the techniques describedherein.

As an example, referring to FIG. 14 and FIG. 15, the REDI softwareapplication 158 of the RTDIP device 540 may employ a combination of3D-sensing and mapping techniques, spatial filtering techniques, imagesegmentation techniques, and color mapping techniques in apre-application prediction mode to allow the user to view how a roomwould appear if painted in a particular color, in accordance with anembodiment.

Security and Law Enforcement Modes

FIG. 16 illustrates an example embodiment of a digitally processed imageof a scene of traffic on a highway, hi-lighting automobiles of aparticular color. Image data acquired by a RTDIP device can applyfilters to display, for example, only cars having a particular color ofblue. Such an embodiment may be useful to law enforcement when lookingfor a vehicle of a particular color in traffic on a busy highway. Theimplementation can be in real-time and the filters can be selectable bythe user. In FIG. 16, the cars of interest (i.e., of a selected color)are outlined by dashed circles.

As an example, referring to FIG. 16, the REDI software application 158of the RTDIP device 510 may employ a combination of spectral filteringtechniques and temporal filtering techniques in a law enforcement modeto allow the user to view automobiles on a highway within a selectedcolor range (e.g., a range of shades of red), in accordance with anembodiment.

FIG. 17 shows an example embodiment of a first image 1710 of a scene ofa store before image processing as well as an example embodiment of asecond image 1720 of the same scene of the store after image processingto hi-light a change from the normal scene. A normal scene of the storemay be an image of the store under certain lighting conditions when nopeople are present. In the processed second image 1720, the colorscorresponding to the normal scene (the background) are muted or reduced(background reduction) whereas the colors corresponding to a new object(e.g., a person) in the store are enhanced. The enhanced object isoutlined by a dashed circle in the second image 1720 of FIG. 17.

As an example, referring to FIG. 17, the REDI software application 158of the RTDIP device 150 may employ a combination of image subtractiontechniques, compression techniques, and IHS transformation techniques ina security mode to allow the user to view recently changed or newobjects within a scene in a store, in accordance with an embodiment, asthe user walks around the store wearing the RTDIP device 150.

In this manner, a security guard monitoring an image or video of thestore may readily see when an intruder is in the store after hours. Inaccordance with an embodiment, an RTDIP device is mounted within thestore producing the second image data 1720. A security guard may belocated remotely from the RTDIP device, watching the second image data1720 on a personal computer that is communicating with the RTDIP devicevia a communication network (e.g., similar to FIG. 8 herein). Inaccordance with an embodiment, the RTDIP device uses, at least in part,image subtraction techniques to discriminate between the normal sceneand a new object in the store.

Other possible uses for RTDIP devices and methods include sign detectionand enhancement, low-light enhancement of a scene, and processing (e.g.,filtering) an image for color blind persons (e.g., allowing a colorblind person to easily see when a traffic light is red or green).Furthermore, RTDIP devices and methods may be used to determine when acoating has fully cured or dried. A coating product may change color asis cures or dries (e.g., designed chromisms). However, such changes incolor may be subtle to the naked eye. An embodiment of an RTDIP devicecan be used to allow a user to clearly discern how far along an appliedcoating is with respect to curing or drying. A designed chromism is asubstance that experiences a reversible change in color resulting from aprocess caused by some form of stimulus (e.g., curing due toevaporation). Designed chromisms may be used in other scenarios, otherthan monitoring curing, as well.

In accordance an embodiment, the RTDIP device is able to record thedigital imaging data (video or still images) for subsequent playback. Inaccordance with an embodiment, the RTDIP device includes alocation-based services capability (e.g., using a GPS receiver) thatprovides for the tagging of digital imaging data (i.e., the correlationof digital imaging data to a location). In this manner, the geographiclocation of where digital imaging data is being acquired may beassociated with the digital imaging data.

Calibration

In accordance with various embodiments, an RTDIP device may becalibrated to provide accurate and reliable use for application,inspection, and prediction scenarios. In one embodiment, a calibrationprocess may correlate the substrate, the coating, and the light source(and other variables such as, for example, lenses) to a standard. Suchcalibration processes may use a stored standard for the substrate orcoating, or may include acquiring and storing a still image. Similarly,a light source determination may be obtained by acquiring and storing astill image of a known standard. Such a standard may be as simple as awhite piece of paper or as precise as a supplied physical standard thatis, perhaps, built into or provided with the device (e.g., a color chipon the inside of a carrying case of the RTDIP device).

For example, different light sources can cause an object to appear tohave different colors, depending on the light source. In accordance withan embodiment, a calibration procedure may be implemented using theRTDIP device to compensate for a light source's affect on the color ofan object. For example, a “true white” color may be digitally stored inthe memory of the RTDIP device that represents what a truly white objectwould look like under substantially ideal lighting conditions (i.e.,lighting uniformly providing all colors in the visible spectrum). Inthis manner, the RTDIP device can “know” what a “truly white” objectlooks like under ideal lighting conditions.

Next, an image may be acquired using the RTDIP device, under current(e.g., non-ideal) lighting conditions, of a white piece of paper or someother physical standard object that is known to be white. The RTDIPdevice can then compare the stored “true white” color to the color ofthe acquired image of the white object under the current, non-ideallighting conditions, to generate a compensation value. This compensationvalue may subsequently be applied to acquired images of a substrate or acoating under the current lighting conditions to compensate for thenon-ideal lighting conditions.

In this manner, digital imaging data being representative of the truecolors of the substrate or the coating may be generated by the RTDIPdevice. Once this calibration for lighting conditions is achieved,subsequent image processing of the acquired digital imaging data may beperformed to provide better discernment between colors in the digitalimaging data (e.g., to more readily discern between coatingthicknesses).

As another example of calibration, when a candidate substrate is aboutto be inspected for contamination before coating, the RTDIP device canprovide a loaded standard of what an uncontaminated (ideal) substratelooks like. The loaded standard of the ideal substrate may be derivedfrom acquiring digital imaging data of an clean, uncontaminatedsubstrate under “ideal” lighting conditions, for example. Anysubsequently acquired images of a candidate substrate, possibly havingcontamination, may be compared to the loaded standard to generatedifference data. The difference data can be used by the RTDIP device tocreate an image for display that shows where contamination exists on thecandidate substrate.

Furthermore, once the candidate substrate is cleaned and determined tobe free of contamination, an image of that clean candidate substrate maybe acquired under the current lighting conditions and compared to theloaded standard to determine a compensation value that may subsequentlybe applied to acquired digital imaging data as the candidate substrateis being coated. In this manner, compensation for differences in coatingcolor due to the underlying substrate may achieved and accurateestimates of coating thickness may be determined.

As a further example of calibration, post-application inspection may beperformed long after (e.g., years after) a coating has been applied to asubstrate. An image of a cured coating may be acquired by an RTDIPdevice shortly after the coating has been applied. Another image of thecoating can be acquired much later and compared to the original coating.The coating may be designed to have chromic characteristics such thatthe color of the coating may change with pH, abrasion, temperature, orsome other environmental parameter. For example, a coating may bedesigned to change color when corrosion develops under the coating(e.g., between the coating and the substrate). An RTDIP device may beconfigured to compare the original image (e.g., acquired years earlier)to the current image to detect and enhance such a change in color due tocorrosion, allowing an inspector to determine any developing corrosionproblems, even though the substrate is still coated.

In accordance with specific embodiments, a spectral response obtainedfrom an enhanced image may include a color of a coating material that isa plural component system. The plural component system includes two ormore components that are combined (e.g., mixed) together to produce thecoating material (e.g., paint). In a first embodiment, the componentsare combined together at a factory that manufactures the coatingmaterial. The factory might have a proportioner (proportioning device)that proportions the components to provide a predetermined (prescribed,desired) volumetric mix ratio. The proportioner might have a purelymechanical mechanism for controlling mix ratio (component proportions),or might be processer-controlled (e.g., computer controlled). In asecond embodiment, the two or more components are provided as a “kit” bythe manufacture and combined (e.g., mixed) together at a job site by auser (person) that will apply (e.g., spray) the coating onto thesubstrate (e.g., wall surface). In this second embodiment, a curingprocess might start when the components are combined (e.g., mixedtogether), giving the user a limited amount of time to apply the coatingmaterial (e.g., mixture) to the substrate before the coating sets.Applying the mixture might be by means of a sprayer. The sprayer mightreceive (e.g., through a tube) the mixture from an outlet of aproportioner (proportioning device). The proportioner might have two ormore inlets to receive the two or more components. Each component mightbe contained in a component container (e.g., pail), from which therespective component is drawn through a tube into a respective inlet ofthe proportioner. The proportioner might have a dial with which the usersets a predetermined (prescribed, desired) mix ratio that is specifiedby the manufacturer. The proportioner might have a purely mechanicalmechanism for controlling mix ratio (component proportions), or might beprocesser-controlled (e.g., computer controlled). In either the first orsecond example mentioned above, the components might exhibit metamerism,in that they are metamers of each other, in that their respective colorsappear to be the same when viewed under a first light source and appearto be different when viewed under a different second light source.Accordingly, first spectral data acquired from the coating under thefirst light source and second spectral data acquired from the coatingunder the second light source can be used (e.g., through formula ortable lookup) to determine the actual mix ratio. Spectral data from morethan two light sources may be used (e.g., through formula or tablelookup) to determine the actual mix ratio, which is especially usefulfor a multi-component system of more than two components.

When multiple components are mixed/combined to form the coatingmaterial, a specific mix ratio is required to achieve the desiredperformance. The coating material exhibits a color different than thoseof the components, and the color of the coating material is related tothe mix ratio of the components. In accordance with one embodiment, afirst component has a color A and a second component has a color B. Whenthese two components are mixed/combined in a coating material, thecoating material may exhibit a color C which can be visually enhancedand quantified with the REDI technology described herein and correlatedto the mix ratio of the two components.

Referring back to FIG. 2, the functionality provided by the REDIsoftware application 158 can be configured (i.e., programmed) for mixratio determination. In accordance with specific embodiments, thesoftware encoded instructions are configured (i.e., programmed) todetermine an interrelationship between spectral responses and mix ratio.For example, an original image is taken from a self-inspecting materialapplied to a substrate surface. The REDI software application 158performs one or more image enhancement operations to generate anenhanced image. A spectral response with respect to a light source isdetermined from the enhanced image. A measurement of the mix ratio isperformed, and the measured mix ratio is stored together with thespectral response in a data structure in the computer memory 156. Theabove-noted process continues to collect a number of data points ofspectral responses and mix ratios. The software encoded instructions areconfigured (i.e., programmed) to determine a formula indicating theinterrelationship between spectral responses and mix ratios based on thecollected data points, e.g., using a linear regression method. Forexample, the formula indicates that the coating material has a spectralresponse based on a mix ratio, given a particular substrate surface anda particular light source. In accordance with an embodiment, the REDIsoftware application 158 may then be configured to determine a mix ratiousing the formula based on a spectral response obtained from an enhancedimage.

In accordance with some embodiments, the collected data points ofspectral responses and mix ratios may be stored into a database in thecomputer memory 156 which maps spectral responses to mix ratios. TheREDI software application 158 then performs a database inquiry to readout a mix ratio from the database corresponding to a spectral responseof the self-inspecting coating material obtained from the enhancedimage.

In accordance with some embodiments, multiple factors of a singlecoating may be measured independently or in parallel with the REDItechnology. For example, a plural component coating can be designed tohave a spectral response related to film thickness and a spectralresponse related to mix ratio both of which can be measured using theREDI technology. These spectral responses may be dependent on eachother, permitting only one to be measured at a given time, orindependent of each other so that both can be measured in parallel. Inaccordance with certain embodiments, a measured coating thickness and ameasured mix ratio are stored together with the corresponding spectralresponse in a data structure in the computer memory 156. The above-notedprocess continues to collect a number of data points of spectralresponses, coating thicknesses and mix ratios. The software encodedinstructions are configured (i.e., programmed) to determine one or moreformulas indicating the interrelationship between spectral responseswith respect to coating thicknesses and/or mix ratios based on thecollected data points. In accordance with an embodiment, the REDIsoftware application 158 may then be configured to analyze both coatingthickness and mix ratio based on a spectral response obtained from anenhanced image.

In accordance with some embodiments, the collected data points ofspectral responses, coating thicknesses, and mix ratios may be storedinto a database in the computer memory 156 which maps spectral responsesto coating thicknesses and mix ratios. The REDI software application 158then performs a database inquiry to read out a coating thickness and amix ratio from the database corresponding to a spectral response of theself-inspecting coating material obtained from the enhanced image.

In accordance with a specific embodiment, metamers of like color may beincorporated into separate components of a plural component coatingcreating metamerism in the mixed coating that can be used to measure themix ratio of the coating using the REDI technology. For example, colorsof two different components of the coating material may match under aspecific light source, but if the spectral power distributions of thetwo components do not match, the color will not match under other lightsources. When mixed, the resulting metamerism of the coating materialmay be proportionally related to the mix ratio of different componentsin the coating and can be analyzed using multiple light sources andcorrelated for mix ratio determination using the REDI technologydescribed herein. Because the second light source used to measuremetamerism provides a spectral response independent of that of filmthickness, it is possible to measure both the film thickness and the mixratio of the same coating in parallel using the REDI technology.

In accordance with some embodiments, a spectral response obtained froman enhanced image may include a recognizable spectral fingerprint of anon-visible component. When the non-visible component is mixed into oneof the components of a plural component coating material, the coatingmaterial retains a same visible color, and the invisible spectralresponse associated with the spectral fingerprint of the non-visiblecomponent may be enhanced and quantified using the REDI technology andcorrelated to the mix ratio. Because the non-visible component canprovide a spectral response independent of that of film thickness, it ispossible to measure both the film thickness and the mix ratio of thesame coating in parallel using the REDI technology.

The spectral data may be acquired (e.g., by camera) from a coating, of amixture, that is applied to (e.g. sprayed onto) a substrate.Alternatively, the spectral data may be acquired from the mixture whilethe mixture is in a container (e.g., bucket).

In summary, systems and methods providing real-time enhanced digitalimaging for the prediction, application, and inspection of coatings aredisclosed. A real-time digital imaging and processing device providesreal-time image acquisition, processing, and display of acquired digitalimaging data to allow a user to discern coating and/or substratevariations beyond that which can be discerned with the naked eye. Thereal-time digital imaging and processing device may also providepre-coating and post-coating inspection capabilities as well as coatingprediction capabilities.

While the claimed subject matter of the present application has beendescribed with reference to certain embodiments, it will be understoodby those skilled in the art that various changes may be made andequivalents may be substituted without departing from the scope of theclaimed subject matter. In addition, many modifications may be made toadapt a particular situation or material to the teachings of the claimedsubject matter without departing from its scope. Therefore, it isintended that the claimed subject matter not be limited to theparticular embodiments disclosed, but that the claimed subject matterwill include all embodiments falling within the scope of the appendedclaims.

For example, the systems and methods may be implemented on various typesof data processor environments (e.g., on one or more data processors)which execute instructions (e.g., software instructions) to performoperations disclosed herein. Non-limiting examples includeimplementation on a single general purpose computer or workstation, oron a networked system, or in a client-server configuration, or in anapplication service provider configuration. For example, the methods andsystems described herein may be implemented on many different types ofprocessing devices by program code comprising program instructions thatare executable by the device processing subsystem. The software programinstructions may include source code, object code, machine code, or anyother stored data that is operable to cause a processing system toperform the methods and operations described herein. Otherimplementations may also be used, however, such as firmware or evenappropriately designed hardware configured to carry out the methods andsystems described herein.

It is further noted that the systems and methods may include datasignals conveyed via networks (e.g., local area network, wide areanetwork, internet, combinations thereof, etc.), fiber optic medium,carrier waves, wireless networks, etc. for communication with one ormore data processing devices. The data signals can carry any or all ofthe data disclosed herein that is provided to or from a device.

The systems' and methods' data (e.g., associations, mappings, datainput, data output, intermediate data results, final data results, etc.)may be stored and implemented in one or more different types ofcomputer-implemented data stores, such as different types of storagedevices and programming constructs (e.g., RAM, ROM, Flash memory, flatfiles, databases, programming data structures, programming variables,IF-THEN (or similar type) statement constructs, etc.). It is noted thatdata structures describe formats for use in organizing and storing datain databases, programs, memory, or other computer-readable media for useby a computer program.

As an illustration, FIG. 18 depicts at 1802 data structures that can beused within the systems and methods described herein. The datastructures 1802 include a mapping data structure that interrelatesspectral responses with coating thicknesses. The data structures 1802can include separate database fields for storing values of spectralresponses with their associated coating thicknesses. In this way, aparticular spectral response can be used to determine the thickness of aparticular coating. If an exact value cannot be obtained for aparticular spectral response, then interpolation between two of theclosest spectral response values is used to determine a coatingthickness. In another example, the data structures 1802 can store aformula or function to map or interrelate spectral responses withcoating thicknesses.

It should be understood that the data structures 1802 can be extended inmany different ways to suit the application at hand. For example, themapping data structures can be extended as shown at 1902 on FIG. 19. InFIG. 19, the interrelationships between spectral responses and coatingthicknesses are specific to light sources, substrate types, coatingtypes, etc. This can be useful in many different situation, such as tominimize the effect of metamerism where a coating may appear to havedifferent colors under different lighting sources.

As another example, FIG. 20 depicts at 2002 the use of data structurescontaining coatings-related metadata. The coatings-related metadata caninclude capturing along with the image data and spectral response datasuch metadata as the location, orientation, time/date, duration,product, lot number, and device/operator associated with the applicationof a coating upon a substrate.

FIG. 21 depicts at 2102 that the coating-related metadata can be usedfor such purposes as alert and notification operations. For example, ifthe thickness of an applied coating as determined by one or more of theapproaches disclosed herein is out of tolerance, then an alert isdetermined, and notification is sent to one or more personnel, includingthe operator of the coating equipment as well as the supervisor. Themetadata can also be used in an inspection capacity where metadata isused to identify that a particular coating does not have the proper mixratio. For example, the coating can be identified via an opticalidentifier (e.g., a QR code). The color visualization approachesdisclosed herein are used to detect that the mix ratio for the coatingis not proper. This results in an alert notification being sent to abatch mixing computer system to adjust the coating composition to aproper mix ratio. Various other users who receive the alert notificationcan include supervisors and operators of the batch mixing systems.

An example method performed by an electronic device includes storing acorrelation between spectral data values and mix ratio values ofcomponents of a mixture. Spectral data is acquired from a coating, ofthe mixture, applied to a substrate. A mix ratio of the coating isdetermined from the acquired spectral data based on the storedcorrelation.

The correlation might comprise a formula defining mix ratio as afunction of spectral data, such that the mix ratio is determined fromthe coating's spectral data using the formula. Alternatively, thecorrelation might comprise a table of data points in which each datapoint includes a mix ratio value and a corresponding spectral datavalue, such that the mix ratio is determined from the acquired spectraldata by a table lookup. Before determining the mix ratio, the acquiredspectral data might be adjusted by enhancing spectral responsedifferentiation of the acquired spectral data. Acquiring the spectraldata and determining the mix ratio might be performed while the mixtureis being applied to the substrate, or after the mixture has cured on thesubstrate.

In an embodiment, the stored correlation might indicate both (i) a firstrelationship between mix ratio values and spectral data values based ona first light source and (ii) a second relationship between mix ratiovalues and spectral data values based on a second light source. Thespectral data may be acquired under a particular light source, and themix ratio may be determined based on the first relationship if theparticular light source corresponds to the first light source and basedon the second relationship if the particular light source corresponds tothe second light source.

In an embodiment, the stored correlation might relate mix ratio valuesto a combination of first spectral data relating to a first lightingsource and second spectral data relating to a second lighting source.The acquired spectral data includes first acquired spectral dataacquired using the first light source and second acquired spectral dataacquired using the second light source. The mix ratio is determinedbased on correlation and the first and second acquired spectral data.The coating includes first and second metamers that visually appear tohave a same color but whose spectral distributions differ. Thedetermined mix ratio is indicative of a concentration ratio of the firstmetamer relative to the second metamer.

In an embodiment, the spectral data values include spectral data valuesfor different colors. The spectral data values also includes spectraldata values for wavelengths that are outside the visible spectrum. Inresponse to determining that the mix ratio is out of tolerance, an alertis sent to a batch mixing system to adjust a coating composition of acoating being prepared by the batch mixing system.

In an embodiment, the correlation is a first correlation, and the methodfurther includes storing a second correlation between spectral datavalues and coating thickness values, and determining a thickness of thecoating based on the second correlation and the acquired spectral data.

In an embodiment, the coating is applied to the substrate by sprayingthe mixture onto the substrate. The acquiring of the spectral data anddetermining of the mix ratio is performed while the coating on thesubstrate is still liquid. A determination, based on the determinedthickness, is made whether to continue spraying the mixture to increasethickness of the coating.

In an embodiment, the coating is a first coating of the mixture, theacquired spectral data is first spectral data, and the correlation is afirst correlation between spectral data values and mix ratio values andrelates to a predetermined coating thickness. The method includesstoring a second correlation between spectral data values and thicknessvalues that depends on mix ratio. The first coating of the mixture isapplied at the predetermined coating thickness. First spectral data fromthe first coating is acquired. The mix ratio of the first coating isdetermined from the acquired first spectral data based on the firstcorrelation. A second coating of the mixture is applied at anundetermined thickness to the substrate. The second spectral data isacquired from the second coating. The thickness of the second coating isdetermined based on the second correlation and the acquired spectraldata. In the above example, the first spectral data is obtained from acoating, of the mixture, at a predetermined thickness. Alternatively,the first spectral data might be acquired from the mixture while themixture is pooled in a container and has a depth (between top and bottomof the mixture in the container) that provides the same spectral data asif the mixture had infinite depth.

In an embodiment, while the coating is being applied to the substrate,the electronic device is supported by a person that is applying thecoating to the substrate. The correlation might be generated by, foreach of multiple coatings, acquiring spectral data of the respectivecoating and measuring a thickness of the respective coating, and thengenerating the correlation, as a formula or database table thatcorrelates spectral data to coating thickness.

The systems, methods, software instructions may be provided on manydifferent types of computer-readable storage media including computerstorage mechanisms (e.g., non-transitory media, such as CD-ROM,diskette, RAM, flash memory, computer's hard drive, etc.) that containinstructions (e.g., software) for use in execution by a processor toperform the methods' operations and implement the systems describedherein.

The computer components, software modules, functions, data stores anddata structures described herein may be connected directly or indirectlyto each other in order to allow the flow of data needed for theiroperations. It is also noted that a module or processor includes but isnot limited to a unit of code that performs a software operation, andcan be implemented for example as a subroutine unit of code, or as asoftware function unit of code, or as an object (as in anobject-oriented paradigm), or as an applet, or in a computer scriptlanguage, or as another type of computer code. The software componentsand/or functionality may be located on a single computer or distributedacross multiple computers depending upon the situation at hand.

It should be understood that as used in the description herein andthroughout the claims that follow, the meaning of “a,” “an,” and “the”includes plural reference unless the context clearly dictates otherwise.Also, as used in the description herein and throughout the claims thatfollow, the meaning of “in” includes “in” and “on” unless the contextclearly dictates otherwise. Finally, as used in the description hereinand throughout the claims that follow, the meanings of “and” and “or”include both the conjunctive and disjunctive and may be usedinterchangeably unless the context expressly dictates otherwise; thephrase “exclusive or” may be used to indicate situation where only thedisjunctive meaning may apply.

The invention claimed is:
 1. A method comprising: storing a correlationbetween spectral data values and mix ratio values of components of amixture; acquiring spectral data from a coating, of the mixture, that isapplied to a substrate; and determining, from the acquired spectraldata, a mix ratio of the components that are in the coating based on thecorrelation; wherein the mix ratio is used by a batch mixing system toprepare a new coating mixture, wherein the storing, acquiring anddetermining are performed by an electronic device.
 2. The method ofclaim 1, wherein the correlation comprises a formula defining mix ratioas a function of spectral data, and the mix ratio is determined from thecoating's spectral data using the formula.
 3. The method of claim 1,wherein the correlation comprises a table of data points in which eachdata point includes a mix ratio value and a corresponding spectral datavalue, and the mix ratio is determined from the acquired spectral databy a table lookup.
 4. The method of claim 1, further comprising, beforedetermining the mix ratio, adjusting the acquired spectral data byenhancing spectral response differentiation of the acquired spectraldata.
 5. The method of claim 1, wherein determining the spectral dataand determining the mix ratio is performed while the mixture is beingapplied to the substrate.
 6. The method of claim 1, wherein acquiringthe spectral data and determining the mix ratio is performed after themixture has cured on the substrate.
 7. The method of claim 1, wherein:the stored correlation indicates (i) a first relationship between mixratio values and spectral data values based on a first light source and(ii) a second relationship between mix ratio values and spectral datavalues based on a second light source; the acquired spectral data isacquired under a particular light source; and the mix ratio isdetermined based on the first relationship if the particular lightsource corresponds to the first light source and is based on the secondrelationship if the particular light source corresponds to the secondlight source.
 8. The method of claim 1, wherein: the stored correlationrelates mix ratio values to a combination of first spectral datarelating to a first lighting source and second spectral data relating toa second lighting source; the acquired spectral data includes firstacquired spectral data acquired using the first light source and secondacquired spectral data acquired using the second light source; and themix ratio is determined from the first and second acquired spectral databased on the stored correlation.
 9. The method of claim 8, wherein: thecoating includes first and second metamers that visually appear to havea same color but whose spectral distributions differ; and the determinedmix ratio is indicative of a concentration ratio of the first metamerrelative to the second metamer.
 10. The method of claim 1, wherein thespectral data values include spectral data values for different colors.11. The method of claim 1, wherein the spectral data values includespectral data values for wavelengths that are outside the visiblespectrum.
 12. The method of claim 1, further comprising: determiningwhether the mix ratio is out of tolerance; and in response todetermining that the mix ratio is out of tolerance, sending, an alert tothe batch mixing system to adjust a coating composition of the newcoating being prepared by the batch mixing system.
 13. The method ofclaim 1, wherein the correlation is a first correlation, and the methodfurther comprises: storing a second correlation between spectral datavalues and coating thickness values; and determining, by the electronicdevice, a thickness of the coating based on the second correlation andthe acquired spectral data.
 14. The method of claim 13, wherein: thecoating is applied to the substrate by spraying the mixture onto thesubstrate; acquiring of the spectral data and determining of the mixratio is performed while the coating on the substrate is still liquid;and the method further comprises determining, based on the determinedthickness, whether to continue spraying the mixture to increasethickness of the coating.
 15. The method of claim 1, wherein the coatingis a first coating of the mixture, the acquired spectral data is firstspectral data, the correlation is a first correlation between spectraldata values and mix ratio values and relates to a predetermined coatingthickness, and the method further comprises: storing a secondcorrelation between spectral data values and thickness values thatdepends on mix ratio; applying the first coating of the mixture at thepredetermined coating thickness; acquiring first spectral data from thefirst coating; determining the mix ratio of the first coating from theacquired first spectral data based on the first correlation; applying asecond coating of the mixture, at an undetermined thickness, to thesubstrate; acquiring second spectral data from the second coating; anddetermining the thickness of the second coating based on the secondcorrelation and the acquired second spectral data.
 16. The method ofclaim 1, wherein, while the coating is being applied to the substrate,the electronic device is being supported by a person that is applyingthe coating to the substrate.
 17. The method of claim 1, furthercomprising, before storing of the correlation, generating thecorrelation by: for each of multiple coatings, acquiring spectral dataof the respective coating, and measuring a thickness of the respectivecoating; and generating the correlation as a formula or database tablethat correlates spectral data to coating thickness.
 18. The method ofclaim 1, wherein curing of the mixture is initiated when the componentsare mixed together.
 19. An electronic device comprising: anon-transitory data storage medium configured to store a correlationbetween spectral data values and mix ratio values of components of amixture; a camera configured to acquire spectral data from a coating, ofthe mixture, that is applied to a substrate; and a processor configuredto determine, from the acquired spectral data, a mix ratio of thecomponents that are in the coating based on the stored correlation;wherein the mix ratio is used by a batch mixing system to prepare a newcoating mixture.
 20. A non-transitory processor readable storage mediumstoring program instructions configured to be executed by a processorto: store a correlation between spectral data values and mix ratiovalues of components of a mixture; receive, from a camera, acquiredspectral data from a coating, of the mixture, that is applied to asubstrate; and determine, from the acquired spectral data, a mix ratioof the components that are in the coating based on the storedcorrelation; wherein the mix ratio is used by a batch mixing system toprepare a new coating mixture.